packages_to_check <- c("stars", "httr", "jsonlite", "tmap", "basemaps", "sp", "sf")
# Check and install packages
for (package_name in packages_to_check) {
if (!package_name %in% rownames(installed.packages())) {
install.packages(package_name)
cat(paste("Package", package_name, "installed.\n"))
} else {
cat(paste("Package", package_name, "is already installed.\n"))
}
library(package_name, character.only = TRUE)
}
#in case tmap does not install
#remotes::install_github('r-tmap/tmap')MODIS NRT Global Flood Product
LANCE MODIS Near Real Time (NRT) Global Flood Product
Overview
In this lesson, you will use R to take a closer look at the data from the LANCE MODIS Near Real Time (NRT) Global Flood Product, including learning about what are LANCE and MODIS, and the NRT Flood products available. You will then learn to select, download, and visualize one of the NRT Flood layers…
Coding Review
This lesson uses the R language and environment. R is a popular language used for statistical computing and graphics.
Learning Objectives
After completing this lesson, you should be able to:
Determine what NRT raster data is available by navigating the LANCE website.
Read a tile map and select a raster tile to download based on a point of interest.
Download near-real-time raster data using the application programming interfaces (APIs) with the GET HTTP request method (Wilson 2024).
View the downloaded raster data to quickly preview.
Classify and place on a map the NRT flood raster data to determine areas with unusual flooding.
Introduction
Atmospheric circulation, water evaporation, and their interactions with land surfaces can impact a region’s rainfall variability. For example, California’s winter is correlated to ocean evaporation near the west coast and eastern North Pacific, and ocean evaporation is a strong factor in increased flooding in the region (Wei et al. 2016). Additionally, drought in the region is associated to high pressure systems off the U.S. west coast, with studies finding that the high pressure system is linked to the Pacific sea surface temperature anomalies, and exacerbated by high evaporation over land due to high temperatures.
It is critical to understand the water cycle and how flooding events develop, particularly as climate change intensifies extreme weather events, because the impacts of flooding can be a risk to human life, and can disrupt infrastructure, agriculture, and natural habitats.
The MODIS/Aqua+Terra Global Flood Product L3 Near Real Time (NRT) 250m Global Flood Product (MCDWD_L3_NRT) (beta) provides daily maps of flooding globally. The product is provided over 3 compositing periods (1-day, 2-day, and 3-day) to minimize the impact of clouds and more rigorously identify flood water (the best composite will depend on the cloudiness for a particular event) (Lin et al. 2019)
What are MODIS and LANCE?
MODIS
The Moderate Resolution Imaging Spectroradiometer (MODIS) is a NASA Earth Observing System (EOS) satellite-based sensor system that creates data products including land surface temperatures, land surface reflectance, radiances, clouds, aerosols, water vapor, active fire, snow cover, sea ice measurements, and other factor information. The MODIS Near Real-Time (NRT) data includes the Flood product which is a daily ~250 m resolution product showing flood and surface water detected from the twice-daily overpass of the MODIS optical sensors.
The satellite data is readily available shortly after it is acquired by the MODIS instrument on board the Terra and Aqua satellites. This space-based instrument distinguishes 36 spectral bands and groups of wavelengths which helps map the extent of snow and ice caused by winter storms and frigid temperatures. Initially, the water-detecting algorithm is applied to both MODIS observations (Terra and Aqua). Due to cloud and terrain shadows create false positives.
To minimize errors, the product is generated with three different compositing periods (1-day, 2-day, and 3-day) to compare results and decide which product has better coverage for the event. Further, they have to differentiate floods from expected surface water through the use of MODIS Land Water Mask (MOD44W), which uses a decision tree classifier trained with MODIS data to produce a global water mask (Carroll et al. 2016).
MODIS adoption aimed to surpass barriers related to satellite data, including cost, delivery timelines, limited formats, and the need for technical expertise. The transition to GFIMS establishes an operational system at FAO, ensuring continuity in meeting NASA data-user needs (Lin et al. 2019).
LANCE
The Land, Atmosphere Near Real-time Capability for EOS (LANCE) is a NASA initiative that provides near real-time access to satellite data, including MODIS. It allows users to access the latest data within a few hours of satellite overpass, enabling rapid responses to environmental events such as floods. LANCE is particularly valuable for emergency response teams and researchers who require up-to-date information for monitoring and assessing natural disasters (Land 2024).
LANCE reduces processing time, allowing for timely computation. Users access the data through platforms like Web Map Service (WMS) and Web Coverage Service (WCS), enabling visualization and analysis for informed decision-making. This NRT approach enhances the speed and accessibility of critical information on vegetation conditions (Zhang et al. 2022).
MODIS NRT Flood MCDWD Data Products
Data Information
The main landing pages for the MODIS NRT Global Flood Product:
The MODIS/Terra+Aqua Combined MODIS Water Detection (MCDWD) algorithm is tailor-made for detecting water bodies using MODIS data obtained from both the Terra and Aqua satellites. This algorithm employs various bands and spectral information to effectively identify and categorize water bodies. This enhances the accuracy and reliability of the flood product generated (Slayback 2023).
The MODIS Near Real-Time (NRT) Flood dataset offers multiple products, each accompanied by corresponding layers. The specific layers depend on the temporal aggregation:
MCDWD_F1_L3_NRT (1-Day product) This product type is the most basic level and provides binary information about water occurrence. Pixels are classified as either containing water or not, offering a simple way to identify flooded areas.
MCDWD_F1CS_L3_NRT (1-Day CS): F1CS has a cloud shadow mask applied on the version of the MCDWD_F1_L3_NRT product
MCDWD_F2_L3_NRT (2-Day): F2 provides additional information by categorizing water occurrences into three categories: no water, low-confidence water, and high-confidence water. This classification allows for a more nuanced understanding of the extent of the flood and its associated confidence levels.
MCDWD_F3_L3_NRT (3-Day): The F3 product, based on the MCDWD (MODIS/Terra+Aqua Combined MODIS Water Detection) is an algorithm that further refines flood mapping by adding additional spectral information. These results create a more accurate representation of water bodies and flooded areas (Slayback 2023).
Exploring the Data
For this exercise, we will be using the MCDWD L3 F3 product: LANCE NRT Flood
First, install and load the R packages required for this exercise:
Check what days are available for the MCDWD L3 F3 product by going to this link: LANCE NRT Primary Server or Secondary.
Based on availability, edit the year_day variable YYYY-DD. Example: ‘2022-01’
#add the year and date you want to search for (YYYY-DD, 2022-01)
year_day <- '2024-076'Determine tiles of interest:
Based on availability, edit the tile_code variable:
#add tile code from the map above (written as h00v00)
tile_code <- 'h05v05'This is the NRT Flood F3 (MCDWD_L3_F3) API URL:
# Primary Server
API_URL <- paste0('https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=MCDWD_L3_F3_NRT&archiveSets=61&temporalRanges=')
# if the primary server is down, the secondary server may be available:
#API_URL <- paste0('https://nrt4.modaps.eosdis.nasa.gov/api/v2/content/details?products=MCDWD_L3_F3_NRT&archiveSets=61&temporalRanges=')We can combine the API URL above with the year_day provided and print the available datasets:
#pasting together URL and year_day
url <- paste0(API_URL, year_day)
print(url)[1] "https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=MCDWD_L3_F3_NRT&archiveSets=61&temporalRanges=2024-076"Loading the Data
NASA earthdata access requires a nasa username and token. How to generate User Token.
Access the NASA Earthdata with the GET function. Replace USER_TOKEN with the token you generated:
# Define headers (include your token)
headers <- c(
Authorization = "Bearer USER_TOKEN"
)
# Make GET request
response <- httr::GET(url, httr::add_headers(.headers=headers))Check the response status from the GET function:
responseResponse [https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=MCDWD_L3_F3_NRT&archiveSets=61&temporalRanges=2024-076]
Date: 2024-03-16 03:09
Status: 200
Content-Type: application/json;charset=UTF-8
Size: 113 kBOut of the response from the server, we’ll check if the response was a success with if (http_status(response)$category == "Success"). If this statement is true, then the content will be assigned to the variable data in JSON format, which is then parsed to a data frame using data_parsed <- jsonlite::fromJSON(data). The data frame contains data_parsed$content, a column with content. We filter the content by tile code using the command content_items <- data_parsed$content[grepl(tile_code, data_parsed$content$name, ignore.case = TRUE), ] and add the results to a data frame.
if (httr::http_status(response)$category == "Success") {
# Read the JSON content into a data frame
df <- httr::content(response, as = "parsed", simplifyVector = TRUE)
df <- df$content
} else {
print("Request failed with status code", httr::http_status(response)$status_code)
}
names(df) [1] "archiveSets" "cksum" "collections" "dataDay"
[5] "downloadsLink" "fileId" "md5sum" "mtime"
[9] "name" "products" "resourceType" "self"
[13] "size" Search the Data Frame and subset the rows that contain tile_code in the downloadsLink column.
# Subset rows where the downloadsLink column contains the tile code
subset_df <- df[grepl(tile_code, df$downloadsLink), ]
subset_df archiveSets cksum collections dataDay
117 61 1740197482 modis-nrt-c6.1 2024-075 = 2024-03-15
downloadsLink
117 https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/archives/MCDWD_L3_F3_NRT.A2024075.h05v05.061.tif
fileId md5sum mtime
117 2171566441 02a2427756b610fc611229faca969e02 1710555446
name products resourceType
117 MCDWD_L3_F3_NRT.A2024075.h05v05.061.tif MCDWD_L3_F3_NRT File
self size
117 /api/v2/content/details/MCDWD_L3_F3_NRT.A2024075.h05v05.061.tif 526600If there is only 1 row, select the string ‘downloadsLink’ column:
download_link <- subset_df$downloadsLink
print(download_link)[1] "https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/archives/MCDWD_L3_F3_NRT.A2024075.h05v05.061.tif"Read the Data
Use the “read_stars()” function from the “stars” R Library to read the geoTiff raster.
The raster is assigned to the “raster_df” variable:
raster_df <- stars::read_stars(download_link)Set the Coordinate Reference System (CRS) to a new raster “my_raster” to plot it in a map. For example, Web Mercator EPSG:3857. Use the st_transform() function:
raster_df <- sf::st_transform(raster_df, 3857)Now the raster data should be stored as a variable “my_raster” with the CRS set to Web Mercator EPSG:3857
Visualizing NRT Flood Data
Plot the raster to quickly view it:
plot(raster_df, axes = TRUE)
Create NRT Flood Plot with Classification
Refer to the MODIS NRT Global Flood Product User Guide for more information.
NRT Flood data has 5 classifications:
| Code | Definition |
|---|---|
| 0 | No Water |
| 1 | Surface Water |
| 2 | Recurring flood4 |
| 3 | Flood (unusual) |
| 255 | Insufficient data |
To view the data in this classification, we’ll need to create a classified legend; however, the NRT Flood data is stored in decimal numbers (aka floating-point). Create class breaks dividing the data by the following breaks, and corresponding colors and labels:
class_breaks <- c(-Inf, 0.1, 1.1, 2.1, 3.1)
colors <- c( "gray", "blue", "yellow", "red")
labels = c("0: No Water", "1: Surface Water", "2: Recurring flood", "3: Flood (unusual)")Add a title for the plot that includes the year, day of year, and tile code:
title = paste("Near Real-Time Flood F3", year_day, tile_code)Generate a basemap from Esri Imagery
To generate a basemap that shows the location of our raster, we must first create a bounding box to match raster_df.
bbox <- sf::st_bbox(raster_df)The basemap_stars() function from the stars library allows us to access Esri imagery layers. We choose “world_imagery” as our background and assign it to the object bm_m.
bm_m <- basemaps::basemap_stars(bbox, map_service = "esri", map_type = "world_imagery")Loading basemap 'world_imagery' from map service 'esri'...The st_rgb function lets us turn the RGB stars item into a single image
bm_m <- stars::st_rgb(bm_m)Plot basemap and NRT Flood data
Generate a plot from the tmap library using the tm_shape() function. We will plot the basemap and the raster_df items.
## tmap mode set to "plot"
tmap::tmap_mode("plot")
## tmap mode can also be set to "view"
#tmap_mode("view")
#create an object the plots the basemap and the NRT flood raster
#with the tmap library, call the tm_shape() function for the basemap
tm_plot <- tmap::tm_shape(bm_m)+
tmap::tm_raster()+
#create a new tmap shape for the NRT flood raster with style as "cat," meaning categorical.
tmap::tm_shape(raster_df, style="cat")+
#add the classification styling to the raster
tmap::tm_raster( palette = c(colors),
title = title,
breaks = class_breaks,
labels = labels )+
#style the plot
tmap::tm_layout(legend.outside = TRUE) +
tmap::tm_graticules(lines=FALSE)View the plot:
tm_plot
To observe a location closer, we can create a new bounding box using latitude and longitude values, clip the data and replot on a map.
Select from the map four corners in degrees including North/South, East/West information. We use these coordinates to create a matrix of points that represent four corners:
# Define the NWES coordinates
north <- 40 #negative values are used for South
south <- 36 #negative values are used for south
west <- -124 #negative numbers are used for West
east <- -120 #negative numbers are used for West
# Create a matrix with the corner coordinates
corners <- matrix(c(west, south, east, south, east, north, west, north), ncol = 2, byrow = TRUE)We can create a bounding box from these corners. The corners are first placed in a Geographic Coordinate System (GCS) WGS84:
# Create a SpatialPolygons object representing a bounding box
bbox_subset <- sp::SpatialPolygons(list(sp::Polygons(list(sp::Polygon(corners)), ID = "bbox")), proj4string = sp::CRS("+proj=longlat +datum=WGS84"))Finally, these points are projected to a Projected Coordinate System (PCS) 3857 to match our raster:
# Transform to projected coordinate system (e.g., EPSG:3857)
bbox_subset <- sp::spTransform(bbox_subset, sp::CRS("+init=epsg:3857"))
bbox_subset <- sf::st_bbox(bbox_subset)We redo the same process as above to plot the data with a basemap but with a different bounding box:
#generate a new basemap
bm_m <- basemaps::basemap_stars(bbox_subset, map_service = "esri", map_type = "world_imagery")Loading basemap 'world_imagery' from map service 'esri'...#combine RGB bands of the bsemap
bm_m <- stars::st_rgb(bm_m)
## tmap mode set to "plot"
tmap::tmap_mode("plot")
## tmap mode can also be set to "view"
#tmap_mode("view")
#create an object the plots the basemap and the NRT flood raster
#with the tmap library, call the tm_shape() function for the basemap
tm_plot <- tmap::tm_shape(bm_m)+
tmap::tm_raster()+
#create a new tmap shape for the NRT flood raster with style as "cat," meaning categorical.
tmap::tm_shape(raster_df, style="cat")+
#add the classification styling to the raster
tmap::tm_raster( palette = c(colors),
title = title,
breaks = class_breaks,
labels = labels )+
#style the plot
tmap::tm_layout(legend.outside = TRUE) +
tmap::tm_graticules(lines=FALSE)
#View the plot:
tm_plot
In this Lesson, You Learned…
Congratulations! Now you should be able to:
Navigate the LANCE data website and determine what data is available.
Select a tile and date to download NRT data.
Create a GET HTTP request to download near-real-time data.
Plot on a map and classify raster data to determine areas with unusual flooding.
Lesson 4
In this lesson we explored the LANCE MODIS Near-Rear-Time (NRT) Flood dataset. In our next lesson we will think of water at the local level and focuse on New Nork State School Drinking Water datasets .
Lesson 4: New York Sate School Water Quality: Exposure to Lead
References
Carroll, M. L., C. M. DiMiceli, J. R. G. Townshend, R. A. Sohlberg, A. I. Elders, S. Devadiga, A. M. Sayer, and R. C. Levy. 2016. “Development of an Operational Land Water Mask for MODIS Collection 6, and Influence on Downstream Data Products.” International Journal of Digital Earth 10 (2): 207–18. https://doi.org/10.1080/17538947.2016.1232756.
Land, Atmosphere Near real-time Capability for EO (LANCE). 2024. “About LANCE.” January 11, 2024. https://www.earthdata.nasa.gov/learn/find-data/near-real-time/about-lance.
Lin, Li, Liping Di, Junmei Tang, Eugene Yu, Chen Zhang, Md. Rahman, Ranjay Shrestha, and Lingjun Kang. 2019. “Improvement and Validation of NASA/MODIS NRT Global Flood Mapping.” Remote Sensing 11 (2): 205. https://doi.org/10.3390/rs11020205.
Slayback, D. 2023. “Modis NRT Global Flood Product - Earthdata.” https://www.earthdata.nasa.gov/s3fs-public/2023-01/MCDWD_UserGuide_RevC.pdf.
Wei, Jiangfeng, Qinjian Jin, Zong-Liang Yang, and Paul A. Dirmeyer. 2016. “Role of Ocean Evaporation in California Droughts and Floods.” Geophysical Research Letters 43 (12): 6554–62. https://doi.org/10.1002/2016gl069386.
Wilson, James. 2024. “NASA API Overview.” November 30, 2024. https://github.com/wilsjame/how-to-nasa?tab=readme-ov-file.
Zhang, Chen, Zhengwei Yang, Liping Di, Eugene G. Yu, Bei Zhang, Weiguo Han, Li Lin, and Liying Guo. 2022. “Near-Real-Time MODIS-Derived Vegetation Index Data Products and Online Services for CONUS Based on NASA LANCE.” Scientific Data 9 (1). https://doi.org/10.1038/s41597-022-01565-2.